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Session Objectives 



Discuss the context , expectations and practice 
of performance measurement/monitoring and 
evaluation for non- profit service providers 

Provide suggestions on how to improve 
performance monitoring and evaluation 
practice, and to enhance learning about 
programs for both providers and funders 



The Current Context for 
Performance Measurement : 
and Evaluation: Demand 

Within the U.S. Federal Government: 

• The Government Performance and Results Act and 
OMB's directives at the federal level 

• Increasing demands for evaluation placed on a federal 
evaluation workforce that was decimated during the 
1980s 

World Bank, OECD, and Other International Organizations 

"Managing for Results" and State-Stat and Citi-Stat initiatives in 
countries, states and cities 

United Way, Large Foundations and Other Donors 

Evidence-Based Policy/Practice/Management Movement 



The Current Context for 
Performance Measurement and 
Evaluation: Supply 

• The demand for credible evidence on public and non- 
Drofit programs continues to exceed capacity to produce 
t. 

• Despite extremely burdensome performance data 
collection and reporting, there is little documented use of 
the data either by providers or funders beyond basic 
accountability for funds spent. 

• Much skepticism persists that monitoring and 
evaluation actually improves public or non-profit 
services. 

• In the development context, there may be different 
standards and expectations held by funders and service 
providers regarding the validity and reliability of the data. 




First Things First... 

Program Evaluation is: 

The application of systematic analytical 
(social science research) methods to 
address questions about program 
operations and results; and 

Performance Measurement (or Monitoring) is 

The routine measurement of program inputs, 
outputs, and outcomes attributed to a program 
and 

Both involve measurement plus judgment!! 



Measurement Does Not 

Ensure Use! ::# 

Performance Measurement: 

• the routine measurement of program inputs, 
outputs, or outcomes in order to monitor programs 
services and results, but 

Performance Management: 

• Using performance measures to inform 
management decision making, planning, resource 
allocation, and evaluation activities. 



Ideally 



Good performance measures: 

• center on desired outcomes; 

• are relevant and useful to managers and 
stakeholders; and 

• provide valid and reliable information. 

Good performance management: 

• connects planning, program management and, 
perhaps, funding 10 performance; 

• nforms management (does not replace role of 
eaders); and 

• assists in coordination of support systems. 



Theory Underlying Program 
Evaluation Practice 



• Evaluation and measurement of programmatic 
performance of programs are undertaken in 
order to improve programs and their 
outcomes -- through providing useful and timely 
information about programs. 

• Typically performance data are collected 
routinely, such as annually, and can address 
questions about quantity and perceived quality, 
e.g., "how many clients were served", and "how 
satisfied were they with the services?" 
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Theory Underlying Program 
Evaluation Practice, cont. 



A more extended effort to evaluate programs is 
needed to answer "Why" and "How" and "With 
what result" questions about services 

Both performance data and evaluation studies 
undertaken can be useful in both performance 
management and in the exercise of 
accountability as long as there are performance 
data deemed both credible and useful 



Learning how to ensure data are credible and 
useful takes time 



Potential Outcomes of 
Programmatic Measurement and 

Reporting 




Ongoing Refinement in 
Performance Measurement 
and Reporting 



Informed Program 
Management 



Informed Exercise 
of Oversight 



4 



Improved Link Between 
Performance and Allocation 
of Resources. 







Knowledge of Program, 
Process and Results 










What Are Typical Barriers to :: 
Evaluation Use and Learning 
within Non-profit Service 

Providers?? 

Lack of clarity to providers regarding; 

• The espoused and actual use of data 

• Who will bear the burden and costs of data collection and 
analysis 

The level of comfort with evaluation and measurement within the 
organizational culture 

• Insufficient clarity and consistency in the priority given evaluation 
from the leadership 

• Clarity in Vertical and Horizontal Communications 

• Workforce Stability 

• Level of comfort with Quantitative Analyses 

• Receptivity to Organizational learning 

Lack of clarity in the theory of change underlying sometimes quite 
complex social programs 

Inability to ensure the validity and reliability of the data collected 



How Can we Facilitate 

Learning? :: 

We examined the experience of service providers in two 
development contexts, Egypt and Colombia, to assess 
how the typical barriers to use and learning may be 
addressed. 

Our study addressed the following key questions: 

• What is current practice in performance reporting? 

• What is current practice in use of performance data by the 
service providers and the funders? 

• What sorts of performance measurement tools and 
measures are most useful to the service providers and 
funders? 

• What sorts of changes do service providers recommend to 
increase the use of performance measurement? 
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Thank you for your interest! 



Email: 

Professor Kathryn Newcomer at 

newcomer@qwu.edu 



